All Systems will be Gamed: Exploitive Behavior in Economic and Social Systems

After the 2008 Wall Street crash, it became clear to economists that financial systems, along with other social and economic systems, are not immune to being manipulated by small groups of players to their own advantage. Given this, two natural questions arise. For a particular policy design or proposed economic system, can such manipulation be foreseen in advance and possibly prevented? And can we design methods—possibly automatic ones—that would test proposed policy systems for possible failure modes and for their vulnerability to possible manipulation, and thereby prevent such behavior in the future? The paper argues that exploitive behavior within the economy is by no means rare and falls into specific classes; that policy studies can be readily extended to investigate the possibility of the policy’s being “gamed”; and that economics needs a strong sub-discipline of failure-mode analysis, parallel to the successful failure-mode-analysis disciplines within structural engineering and aircraft design.